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Abstract — This paper provides a mathematical framework for 
Fisher information analysis for inverse problems based on Gaus- 
sian noise on infinite-dimensional Hilbert space. The covariance 
operator for the Gaussian noise is assumed to be trace class, and 
the Jacoblan of the forward operator Hilbert-Schmidt. We show 
that the appropriate space for defining the Fisher information 
is given by the Cameron-Martin space. This is mainly because 
the range space of the covariance operator always is strictly 
smaller than the Hilbert space. For the Fisher information to 
be well-defined, it is furthermore required that the range space 
of the Jacoblan is contained in the Cameron-Martin space. In 
order for this condition to hold and for the Fisher information to 
be trace class, a sufficient condition is formulated based on the 
singular values of the Jacoblan as well as of the eigenvalues of the 
covariance operator, together with some regularity assumptions 
regarding their relative rate of convergence. An explicit example 
is given regarding an electromagnetic inverse source problem 
with "external" spherically isotropic noise, as well as "internal" 
additive uncorrelated noise. 

Index Terms — Inverse problems, Fisher information, Cramer- 
Rao lower bound, trace class operators. 



I. Introduction 

An important issue within inverse problems and imaging 
[15] is to develop an appropriate treatment of the uncertainty 
associated with the related quantity of interest (the state), 
and the observation process, respectively. Statistical methods 
provide many useful concepts and tools in this regard such as 
identifiability, sufficient statistics, Fisher information, statisti- 
cally based decision rules and Bayesian estimation, see e.g., [5, 
12,33]. The Bayesian view is e.g., based on the assumption 
that uncertainty always can be represented as a probability 
distribution. Hence, statistical information about the state can 
be retrieved by means of Bayesian filtering such as Kalman 
filters, extended Kalman filters and particle filters, see e.g. [12, 
33]. 

On the other hand, the conditional Fisher information can 
be a useful "deterministic" analysis tool in various inverse 
problem applications such as with material measurements 
[32], parametric shape estimation [35], sensitivity analysis 
and preconditioning in microwave tomography [23,24], and 
in electrical impedance tomography [22]. The Cramer-Rao 
lower bound can furthermore be used to quantify the trade- 
off between the accuracy and the resolution of a linear or 
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non-linear inverse problem [7, 22-24, 26], and it complements 
the deterministic upper bounds and the L-curve techniques 
that are employed with linearized inversion, see e.g., [9, 15]. 
The Fisher information is also a tool that can be exploited for 
optimal experimental design, see e.g., [29]. 

The Fisher information analysis has mostly been given in 
a finite-dimensional setting, see e.g., [13,31]. One of the 
very few exceptions can be found in e.g., [34] where the 
Fisher information integral operator for an infinite-dimensional 
parameter function has been employed in the estimation of 
wave forms (or random processes). An infinite-dimensional 
Fisher information operator has previously been exploited in 
the context of inverse problems in e.g., [22,23,25], and the 
connection to the Singular Value Decomposition (SVD) has 
been established. However, so far, a rigorous treatment of 
Gaussian noise on infinite-dimensional Hilbert spaces has been 
lacking. 

A Fisher information analysis for inverse problems based 
on Gaussian noise on infinite-dimensional Hilbert space [2, 
6, 8, 14, 20] requires a careful study of the range spaces and 
the spectrum of the operators related to the covariance of the 
noise, and the Jacobian of the forward operator, respectively. 
It is assumed here that the covariance operator is trace class 
and the Jacobian is Hilbert-Schmidt [30]. Since the range 
space of the covariance operator always is strictly smaller 
than the Hilbert space, the appropriate space for defining the 
Fisher information operator is given by the Cameron-Martin 
space [2,8]. In order for the Fisher information to be well- 
defined, it is furthermore required that the range space of the 
Jacobian is contained in the Cameron-Martin space. In order 
for this condition to hold and for the Fisher information to 
be trace class, it is shown in this paper that it is sufficient 
that the singular values of the Jacobian and the eigenvalues of 
the covariance operator satisfy some regularity assumptions 
regarding their relative rate of convergence. It is also shown 
how the Cramer-Rao lower bound for parameter estimation 
based on finite-dimensional eigenspaces is closely linked to 
the Singular Value Decomposition (SVD), and the truncated 
pseudo-inverse based on the Cameron-Martin space. 

The situation becomes particularly simple if the covariance 
operator and the Jacobian share the same left-singular vectors, 
implying that a sequence of finite-dimensional subspace esti- 
mators (truncated pseudo-inverses) may readily be analyzed. 
In this case, the eigenvalues of the Fisher information is simply 
the ratio between the squared singular values of the Jacobian, 
and the eigenvalues of the covariance operator Two important 
special cases arises: 1) The infinite-dimensional Fisher infor- 
mation operator exists and is trace class, and the corresponding 



pseudo-inverse (and the Cramer-Rao lower bound) exists only 
for finite-dimensional subspaces. 2) The infinite-dimensional 
pseudo-inverse (and the Cramer-Rao lower bound) exists, and 
the corresponding Fisher information operator exists only for 
finite-dimensional subspaces. A concrete example is consid- 
ered regarding an electromagnetic inverse source problem with 
"external" spherically isotropic noise, as well as "internal" 
uncorrected noise. The example illustrates how the singular 
values of the Jacobian, the eigenvalues of the covariance 
operator and the internal noise variance can be analyzed and 
interpreted in terms of the Cramer-Rao lower bound. 

II. Cramer-Rao Lower Bound for Inverse 
Problems 

A. The forward model 

Below, two common situations will be considered in par- 
allel, where the parameter space (describing the quantity of 
interest) is real or complex, respectively. The observation 
process (the measurement) is assumed to be executed in 
the frequency-domain with complex valued data. Hence, an 
inverse problem is considered below where ^ denotes the 
complex valued measurement and 6 the real or complex valued 
parameter function to be estimated. The forward operator is 
defined as the mapping 



il^ie) -.e^n, 



(1) 



where O and H are separable Hilbert spaces [16]. The mea- 
surement space H is complex, and the parameter space 9 is 
either real or complex. The same notation (•,) is employed 
here to denote the scalar products in both spaces. 

Let Re{-} denote the real part of {•}. Note that Rc(^i,^2) 
is a scalar product defined on the elements ^i and ^2 of 
H interpreted as a Hilbert space over the real scalar field. 
In this space, the vectors ^ and i^ are orthogonal, i.e., 
Re(^, i^) = Rei(^,^) ~ 0, and the norm is preserved, i.e., 
Re(^,^) = (^,^) = 1 1 ^11 2. Hence, the Hilbert space V. over 
the complex scalar field with basis vectors {ci} is isomorphic 
with the Hilbert space H over the real scalar field, and with 
basis vectors given by {e^} and {ie^}. 

It is assumed that the forward operator tp{9) is Frechet 
differentiable [15] in a neighborhood of the (background) 
parameter vector 6'b G 6. The first order variation dip can 
then be represented as 



ap^jdo, 



(2) 



where the bounded linear operator J' : 6 ^ "H is the Frechet 
derivative (or Jacobian) of the operator V'(^) evaluated at 6h, 
and S) G Q is an incremental parameter function where 6 — 
9\j + S). Let J* denote the Hilbert adjoint operator to J . It is 
assumed that the Jacobian J is Hilbert-Schmidt [30], so that 
trji/* J'} < 00 and both J and J* are compact. 

For a bounded linear operator, the Hilbert-adjoint operator 
exists and is unique provided that the Hilbert spaces are 
defined over the same (real or complex) scalar field [16]. 
Hence, in the case when 9 is real the Hilbert-adjoint operator 
J* is defined by the property 



and (J«',0 = {Sd,J*C) when 6 is complex. As e.g., if 
the Jacobian J can be represented by a (finite or infinite- 
dimensional) matrix J, the adjoint J*^^ is given by Re{J^^} 
and J^^ with respect to the real and complex spaces 8, 
respectively, where ^ denotes the representation of ^ G H, 
and (•)^ the Hermitian transpose. 

The Singular Value Decomposition (SVD) of the Jacobian 
J can be expressed as follows. Assume first that 8 is complex. 
Let the singular system for the linear compact operator J be 
given by 

{ (4) 

( J*Ui ^ aiVi, 

where {ui] and {vi] are orthonormal systems and (Ji are the 
singular values arranged in decreasing order, and where ai -^ 
as i —7- 00 if the number of non-zero singular values is 
infinite [15]. The operator J can then be represented as 



JS) = y^^aiUi{vi 



i£l 



the adjoint operator J* as 



(5) 



(6) 



iG/ 



and the pseudo-inverse J^ as 



(7) 



i£l 



where / is the index set (finite or infinite) corresponding to 
the non-zero singular values. Here, (ui,^) is replaced for 
Ke{ui,£_) when 8 is real. Note that the expression (7) is 
valid provided that the pseudo-inverse exists, and the series 
J2i l("iiOP/'''f converges [15]. A truncation of (7) using a 
finite number r of non-zero singular values is referred to as 
a truncated pseudo-inverse. The corresponding regularization 
strategy is denoted 



2—1 



0, 



(8) 



where r is the regularization parameter. It is readily seen that 



lim J+JS9 

r—>-oo 



S9, 



(9) 



where S9 belongs to the orthogonal complement of the null 
space of the operator J, i.e., S9 e N{J)-^, see e.g., [15]. 
Note that the SVD gives an orthogonal decomposition of the 
two Hilbert spaces as follows 



8 = span{wJ,e/e7V{J'}, 
•H = span{wJ,e/®AA{J^*}, 



(10) 



Rc(^«,0-(«,J*0, 



(3) 



where span{} denotes the closure of the linear span of the 
vectors in {•} and the direct sum [15]. 



B. Fisher information for the Cameron-Martin space 
Consider the statistical measurement model 



(11) 



where ^ is the measured response, V'(^) the forward model 
and w e H is zero mean complex Gaussian noise. Here, w 
is an observation of a complex Gaussian stochastic process 
defined on the Hilbert space Ti, [2,6,8, 14,20]. Let £ denote 
the expectation operator with respect to the complex Gaussian 
measure /x and B the corresponding covariance operator. It 
is assumed that the positive operator B is trace class so that 
^{Ih^lP} — tr{i?} for any w E H, and hence that B is self- 
adjoint and compact [30]. It follows from the Hilbert-Schmidt 
theorem [30] that there exists a complete orthonormal basis 
{(f>j} for Ti, such that 



A, ■(/),■, 



(12) 



where (pj are the eigenvectors of B and Xj the non-negative 
eigenvalues, and where Xj — > as j — )• oo if the number of 
non-zero eigenvalues is infinite. Let J denote the index set 
(finite or infinite) corresponding to the eigenvectors with non- 
zero eigenvalues Xj > 0. The Hilbert space H is then given 
by the following orthogonal decomposition 



n = span{(j)j}jej®J^{B}. 



(13) 



The linear functionals {£_, w) and (77, w) are complex Gaus- 
sian stochastic variables with zero mean, and with the covari- 
ance properties 



£{{C,w){rj,w)*}^{^,Brj), 



(14) 



for any ^,77 e H, and where (•)* denotes the complex 
conjugate [2,6,8,14,20]. 

The description of the stochastic process on H is now 
complete. However, since the range of the operator B always 
is strictly smaller than H for infinite-dimensional spaces', it 
is not possible to define a non-singular Maximum Likelihood 
(ML) criterion [13,31,34], or trace class Fisher information 
based on the whole of H. Moreover, in order to be able to 
define the Fisher information in trace class, the range of the 
Jacobian J' also needs to be restricted to this smaller subspace. 
The proper subspace for this purpose is the Cameron-Martin 
space H^ [2, 8], which can be constructed as follows. Consider 
the covariance operator B restricted to the range of B, i.e., 



B -.niBj^TZiB}, 



(15) 



then the (self-adjoint) inverse B^^ exists on this subspace 
since B > 0, and hence B^^ > on TZ{B}. 
Define the scalar product 



(e,r7)^ = Rc(e,S-^), 



(16) 



'To see this, assume that TZ{B} = 'H. Now, B is trace class and thus 
bounded, and B^^ exists since _B > 0. It then follows by the bounded inverse 
theorem [16], that B^^ is bounded. Since the composition of a trace class 
operator with a bounded operator is trace class [30], the equality BB~^ = I 
gives a contradiction since the identity operator is not trace class for infinite- 
dimensional spaces. 



if <d is real, and {£,,ri)f^ = (S^^B^^rj) if Q is complex, and 
where £,,1] E TZ{B}. The Cameron-Martin space H^ is the 
completion of Tl{B} with respect to the norm induced by the 
scalar product (^,77)^, see [2,8]. It can be shown [2,8] that 
"Hp is also given by 



n, = {cen\J2h{cf,„o\'<^}, 



A, 



(17) 



Hu 



n{B^'^} 



where n{B} ^n^^^U. 

It is assumed that the range of the Jacobian J is restricted 
as 7^{ J} C niB}, implying that UiJ} C U^ ^. Hence, it 
is sufficient to consider the mapping 

J-.Q^H^,. (18) 

Since "H^ is a Hilbert space on its own right and J a bounded 
linear operator, the Hilbert adjoint operator J* exists and is 
defined by the property 



(^«,Op-(«,j:o, 



(19) 



where S) E <d and £ E 7^^. It is readily seen that J'* ~ 
J*B^^ on Tl{B}, where J* was defined as in (3). 

The Fisher information operator Z : 8 — t- can now be 
defined as 

(20) 



T^2j;j^2J*B-^J, 



when <d is real, and I = J* J when 6 is complex, see also 
[13,22,23,34]. It is assumed that the operator J is Hilbert- 
Schmidt with respect to the Cameron-Martin space "H^, and 
hence that J is compact with respect to the norm induced 
by the scalar product (^, 77)^. The Fisher information operator 
is thus assumed to be trace class. Let /ij and vi denote the 
eigenvalues and the eigenvectors of the Fisher information 
operator Z, respectively, i.e.. 



IVi = jliVi 



(21) 



where the eigenvectors vi are assumed to be orthonormal, i.e., 
{Vi,Vj) = %. 

The Singular Value Decomposition (SVD) of the Jacobian 
J' with respect to the Cameron-Martin space H^, can now 
be expressed as follows. The singular system for the linear 
compact operator J' is given by 



JVi = CTiUj 



J*Ui = aiV^, 



(22) 



where {ui} and {vi} are orthonormal systems and (t^ are 
the singular values arranged in decreasing order, and where 
CTi ^ as i ^ 00 if the number of non-zero singular 
values is infinite [15]. Note that the eigenvalues of the Fisher 
information operator are given by jii — 2(t| when 8 is real, 
and j2i = af when G is complex. 

^The main results of this paper can be generalized to larger class of 
Jacobians J' having the image in Hf^ (and not just in its proper subspace 
Ti{B}. However, this will make the paper more complicated mathematically, 
since one cannot proceed with continuous linear functionals given by the scalar 
product, and measurable linear functionals (square integrable with respect to 
the Gaussian measure /i) have to be used. 



The operator J can be represented as 



is/ 



the adjoint operator J* as 



iei 



and the pseudo-inverse J'^ as 



(23) 



(24) 



(25) 



is/ 



where / is the index set (finite or infinite) corresponding to the 
non-zero singular values. The expression (25) is valid provided 
that the pseudo-inverse exists and the series ^^ \{ui, Ciij.\^ I'^f 
converges [15]. A truncation of (25) using a finite number r of 
non-zero singular values is referred to as a truncated pseudo- 
inverse. The corresponding regularization strategy is denoted 






(26) 



i=l 



where r is the regularization parameter. It is readily seen that 



lim J+JSd = S9, 



(27) 



where W G Af{J)^, see e.g., [15]. Note also that the SVD 
gives an orthogonal decomposition of the two Hilbert spaces 
as follows 



e = spanjwj -gj ® M{J}, 



H^-spanluJ^g^-eAAlJ-;}. 



(28) 



C. The Cramer-Rao lower bound based on trace class oper- 
ators 

Consider the statistical measurement model given by (11). 
Consider also the SVD of the Jacobian J with respect to the 
Cameron-Martin space 'H^ given in (22), where J is evaluated 
at the fixed background parameter vector 6'b, and let 6 ~ 6\y + 
S). Any incremental parameter vector S) <E Q can be uniquely 
decomposed as 



where i <E I. The covariance of the noise term is given by 



1 



Rc(Mi,i? Uj 



Ui,Uj) f^ — dij, 



(32) 



where i,j El and where (14) and (16) have been used. When 
Q is complex, the corresponding discrete measurement model 
is given by 

{B-^u^,0 = {B-^u^M9)) + {B-'u,,w), (33) 
where i E I. The covariance of the noise term is given by 



£{{B-^Ui,w){B-^Uj,w)*} 



(34) 



where i,j E I. 

Assume once again that 6 is real. By using S) — Vjd'dj and 
letting d-dj -^ 0, it is readily seen that 



_d_ 



Rc{B~^u„tP{9)) 
— Ile{B^^Ui, J'vj) — Iie{B^^Ui, ajUj) — d'jSij, 



(35) 

where i,j E I. When 8 is complex, the corresponding 
expression is obtained by deleting the Re{-} operation above. 
Consider now the projection (approximation) of S) onto 
the finite dimensional subspace spanned by the orthonormal 
system {wi}[^i, i.e., 



%■ = y^^'&iVj, 



(36) 



where r is the dimension of the approximating subspace. It 
follows from (35) and the orthogonality of the noise terms 
(32) and (34), that a sufficient statistics [13,31] for estimating 
ddr is given by the discrete measurement model (31), or (33), 
restricted to the index set Ir — {1, . . . ,r}. The corresponding 
unbiased estimator is represented as 



% = y^^'&iVj, 



(37) 



S9 = 39 + 00 = ^^^^ + 90, 
iei 



(29) 



where 39 E M{J)^ and 9o E M{J). The principal parame- 
ters of interest are given by 



I?,:- 



(30) 



where i E I. 

Assume first that is real. Consider the discrete (countable) 
measurement model that is obtained by applying the scalar 
products Yic{B~^Ui,-) to the statistical measurement model 
(11). Note that u, E Tl{J} C Tl{B} so that B'^Ui is 
uniquely defined. The discrete measurement model is hence 
given by 

Rc{B-^u„C} =Rc{B-^u„'il){9)) +Rc{B-^u„w), (31) 



where di are unbiased estimates of the principal parameters 
defined in (30). 

Assume that 9 is real. The Cramer-Rao lower bound [13, 
31,34] for estimating the principal parameters follows then 
from the measurement model (31), (32) and (35), and is given 
by 

£{\d,-'d,\''}>^ = ^, i = l,...,r, (38) 

where jli = 2(t| are the eigenvalues of the Fisher information 
operator defined in (20). A similar expression is obtained when 
& is complex, and jli = af. It follows directly from (38) that 
the mean squared estimation error is lower bounded as 



£{\\ 



r ^' 1 

^'} = Y^£{\i,-i),f}>j2 (39) 



i=l 



which is vaUd for both the real and the complex space Q. Note 
that the truncated Fisher information operator is given by 

r 

Ir=Y,f'^M^^'^)^ (40) 

i=l 

and the corresponding truncated pseudo-inverse 

2:+ = ;^^i),(s„a?), (41) 



i=l 



fj'i 



and the Cramer-Rao lower bound [13, 31, 34] can also be stated 
as 



£{{0l,S9r - S9r){Ser - Ser,92)} > {61,1+62), 



(42) 



where 61,62 E 6. It is immediately observed that the operators 
Ir and Z+ cannot both be trace class in the limit as r ^ 00. 
If the infinite-dimensional Fisher information is trace class, 
then the corresponding pseudo-inverse does not exist, and vice 
verse. 

An interesting interpretation of the Cramer-Rao lower bound 
is obtained as follows. It is noted that the lower bound defined 
in (39) is a function of the spatial resolution, i.e., the subspace 
dimension r. As the desired spatial resolution r increases, the 
estimation accuracy decreases in the sense that the Cramer- 
Rao lower bound increases. The Cramer-Rao lower bound 
gives the best possible (optimal) performance of any unbiased 
estimator related to the non-linear inverse problem at hand. 
Hence, the lower bound defined in (39) gives a quantitative 
interpretation of the optimal trade-off between the accuracy 
and the resolution of an inverse problem. 

The truncated pseudo-inverse (26) gives an example of an 
efficient unbiased estimator of d9r as follows. Consider a linear 
version of (11) 



^^i^{6b) + jse + w. 



(43) 



where the forward operator ip{6) has been replaced by its first 
order approximation, the space 8 is real and the noise is as- 
sumed to have zero mean, i.e., £{w} — 0. The corresponding 
truncated pseudo-inverse is given by 



^ 1 



(44) 



i=l 



and the corresponding parameter estimates are hence given by 

^t = ^Rc{B-^Ui,jg6) + ^Rc{B-^Ui,w). (45) 

The mean value of (45) is given by 






and the variance 



var{^J = var{— Rc(B ^u^, w)} = — ^ = — , 



(46) 
(47) 



where (22) and (32) have been used. A similar result is 
obtained when 8 is complex. Hence, with a linear estimation 



model as in (43), the truncated pseudo-inverse (26) leads to 
an unbiased estimator which is efficient in the sense that it 
achieves the Cramer-Rao lower bound (39), see also [13,31, 
34]. For non-linear estimation problems, it should be noted that 
the Maximum Likelihood (ML) criterion yields an estimation 
error that approaches the Cramer-Rao lower bound as the data 
record is getting large, i.e., the ML-estimate is asymptotically 
efficient, see [13,31,34]. 



D. Conditions for the Jacobian 

A sufficient condition is formulated for the Jacobian J to be 
restricted as Tl{J^} C TZ{B}. This condition also guarantees 
that J' is Hilbert- Schmidt with respect to the Cameron-Martin 
space Hfj,. Note that the result is valid for both real and 
complex spaces 8. 

Proposition II.l. Let the Jacobian J' : ^ H be Hilbert- 
Schmidt with respect to %, with the singular system (4). Let 
the positive, self-adjoint covariance operator _B : "H — > "H 
be trace class, with the singular system (12). Suppose that 
TZ{J} C span{(f)j}j(zj. 

If 



\2 I 

iei jeJ J 



(48) 



then TZ{J^} C TZ{B}, and J is Hilbert-Schmidt with respect 
to the Cameron-Martin space "H^, and 



tr{j; ^} = E '^^^ E X" I ^^' ' "*^ ' 



(49) 



i£l je.J 



Proof: For the operator equation BS, = J 6 to be solvable, 
it is necessary and sufficient that J6 e M{B^+ , and that 



for any € 8, see e.g., [15]. Hence, it is assumed that 
7?.{J} C M{B^^ = span{(j)j}j^j, and it is then sufficient 
to consider (50). By using (5), it follows that 

je.J i iei 

= E T2 I E '^i('^J' "*) ("^^^ ^>P 



jeJ 1 iei 



je.] i Kiel ) Kiel 

jeJ 1 Kiel 



"^>n \\of<^, 



(51) 

for any 6 E Q, and where the Cauchy-Schwartz inequality has 
been used. Hence, n{J} C n{B}. 



Now, 



ie/ 



iei 



i£l iel 

and since Ui e 7^{J'} C Tl{B} C span{(f) j}j^j, 

iei jeJ •' 

iei jeJ -^ 



(52) 



(53) 



A, 
iei jeJ •' 

where the last term converges faster than (48) as Xj ^ 0. □ 
It is easy to find Hilbert- Schmidt operators J' and trace 
class operators B such that the conditions in proposition 
II.l are satisfied. An example can readily be constructed as 
follows. Let the Hilbert- Schmidt operator J' be given, having 
the singular system (4). It is furthermore assumed that the 
singular values decay sufficiently fast so that J^iei V^ "^ °°- 
Let (f)i = Ui for alH e / = J, and define the operator B by 



SC = ^A,0,(0„O, 



iei 



where 



A. 




,q, 



(54) 



(55) 



and where cr^ > and g is a positive integer. Note that 
Bipi = Xi(f)i. Since (pi — Ui for all z e / = J and 7?.{J} C 
span{ui}ig/, it follows directly that 7?.{J} C span{(/)j}jgj. 
The sufficient condition (48) can now be verified as follows: 



oeJ ^- ,e.i ^- 

for all i e /, and 



= ^<oo, 



(56) 



JO 



lei jeJ J 

9 ,2 



lei 



(57) 



< oo, 



i=l 



i=q+l 



which shows that TZ{J} C 7?.{S}, and J is Hilbert-Schmidt 
with respect to the Cameron-Martin space Ti,^. 



The singular values di defined in (22) are given by 

^2 



(Jt 



%2 



!,■ 



,9, 



CT, 



A, 



(58) 



3/2 . ^ 

0-j « > 9, 



and the singular vectors Ui = \/\iUi and Vi — Vi. These 
relations are obtained by using Jvi = (JiUi, J*Ui ~ 



OiVi, B' 



Ai 



j; = TB- 



and 



{y/XiUi,B ^JT^jUj) = {^\iU^,Uj/J~X^) = 5, 



\Ui , Uj J ^ 



It is impossible to have a white noise stochastic process w 
with a trace class identity covariance operator B — I defined 
on infinite-dimensional spaces. However, as the example above 
shows, the stochastic process w can be white with identity 
covariance operator B ~ I on finite-dimensional subspaces. 
In particular, in the example above, let the approximating 
subspace dimension r be fixed and choose q > r such that 



> 0-„,cr, 



3/2 



then the first r singular values and singular 



vectors are given by 



crt. 



1 = 1,.. 



Ui = a^Ui i = l,...,r, 
v, = Vi i = l,...,r, 



(59) 



and except for a scaling, the singular systems (4) and (22) are 
the same for i = 1 r. 



E. Unbounded Fisher information and finite Cramer-Rao 
lower bound 

It is easy to find Hilbert-Schmidt operators J' and trace class 
operators B such that the conditions in proposition II.l are 
not satisfied. In particular, if the condition Il{J} C TZ{H^} is 
not satisfied, then the Cameron-Martin space cannot be used 
as indicated in (18) above. In this case, it may be difficult 
(or technically very complicated) to obtain a useful definition 
of the Fisher information and the Cramer-Rao lower bound. 
However, a simple case which is also very useful, arises when 
the singular vectors Ui of the Jacobian operator J coincides 
with the eigenvectors (pj of the covariance operator B. Hence, 
the following assumptions are made 



(60) 



where i E I. For simplicity, it is also assumed here that the 
space 8 is complex (the case with real 8 is similar). 

Consider now a sequence of finite-dimensional measure- 
ment models based on (11) 




PrC = Pr1p{0) + PrW, 



(61) 



where Pr denotes the projection operator Pr : H =' 
spanjui, . . . , Ur}. The finite-dimensional Jacobian and covari- 



ance operators are given by 

r 
PrJS) = 'y^aiUi{Vi,( 



i=l 



(62) 



PrBPri = Y,^^U^i'^^^0■ 



Since the requirement Tl{PrJ} C TZ{PrBPr} is trivially 
satisfied, all the results of the previous sections apply in this 
finite-dimensional case. In particular, the Fisher information 
operator (20) becomes 



' 2 



(63) 



2 — i 

and the Cramer-Rao lower bound (39) is given by 

£{\\S9,-Wrf}>J2^, (64) 

where jli = af = erf / \i, Ui = ^/XiUi and Vi = Vi, and where 
Sdr and dOr are defined as in (36) and (37), respectively. 

It is easy to find Hilbert-Schmidt operators J' and trace 
class operators B such that the Fisher information operator 
(63) does not converge as r — > oo, but the corresponding 
pseudo-inverse 

A,: 



I+S9^ 






(65) 



is trace class, and the right-hand side of (64) converges. It 
is noted that when erf /Xi -^ oo, it may still be natural to 
organize the eigenvalues according to the magnitude of the 
singular values <Ji defined by the Jacobian J . 

The next section will give an example of an inverse problem 
were the statistical model is given by a physically well- 
motivated external noise source, and where the Fisher infor- 
mation operator (63) does not converge, the pseudo-inverse 
(65) is trace class and the Cramer-Rao lower bound (64) 
is finite as r ^ oo. It should be noted, however, that in 
a realistic, real measurement scenario, internal noise in the 
form of measurement errors are always present. This could 
mean e.g., the addition of uncorrected measurement noise. A 
simple example of this scenario is to consider the addition 
of the noise eigenvalues described in (55), which will yield a 
trace class Fisher information (63), and an infinite Cramer-Rao 
lower bound (64). 

III. An electromagnetic inverse source problem 

WITH spherically ISOTROPIC NOISE 

As an application example of the theory developed in sec- 
tion II, an electromagnetic inverse source problem [17-19,26] 
is considered here, where the observation (or measurement) 
is corrupted by spherically isotropic noise [4, 10], cf.. Fig. 1. 
Here, J is the current source which is contained within a 
sphere of radius ro, -E is the transmitted electric field and Eg 
the spherically isotropic noise. The tangential components of 
the fields are observed at a sphere of radius ri. The objective 
here is to quantify the optimal estimation performance of 



the truncated pseudo-inverse (26) in terms of the Cramer-Rao 
lower bound (39), where S9 = J. 

Below, r, 6 and (p will denote the spherical coordinates, 
and r = rr the radius vector where r is the corresponding 
unit vector. The time convention is defined by the factor c^'"* 
where u is the angular frequency and t the time. Furthermore, 
let k, Co and rjo denote the wave number, the speed of light 
and the wave impedance of free space, respectively, where 
k — w/cq. 



E. 



i 




E 



^ = F + N 



Fig. 1. Illustration of the electromagnetic inverse source problem with 
spherically isotropic noise Eg. 



The transmitted electric field E(r) satisfies the Maxwell's 
equations [11] and the following vector wave equation 



V X V X E{r) - PE{r) = ifcr/o J(r) 



(66) 



and the spherically isotropic noise Es{r) satisfies similarly the 
corresponding source-free Maxwell's equations. The observa- 
tion model (11) is given by 



^ = F{r) + N{f), 



(67) 



where F{f) = f x f x E{rif) and N{r) = —r x r x 
Es(rif) are the tangential components of the transmitted 
electric field E{r) and the noise field Es{r), respectively, as 
they are observed at a sphere of radius ri . 

The space 8 consists of all complex vector fields J{r) 
which are square integrable over the spherical volume Vrg 
with radius tq, and is equipped with the scalar product 



{JuJ2) = 



f Jl{r)-Mr)dv, 



(68) 



where {•}* denotes the complex conjugate. The space Ti, 
consists of all tangential vector fields F (with r ■ F — Q) 
which are square integrable over the spherical surface Sr^ with 
radius ri, and is equipped with the scalar product 






(69) 



A. Forward model based on vector spherical waves 

The electromagnetic fields in a source-free region can be 
expressed in terms of the vector spherical waves defined in 
appendix A, see also [1,3,11,21]. Hence, by considering 
the free-space Green's dyadic defined in (102), the forward 



operator J : Q ^ V. is given by 

2 cx) I 

JJ{r) = -Pr]Q^Y^ ^ JrlmArlm{r) 
T=l 1 = 1 m= — l 

vU^{kr')-J{r')dv', 



(70) 



where Arimi'f^) are the vector spherical harmonics and 
Vrimikr) the regular vector spherical waves defined in (95) 
and (96). Note that the dagger notation {-Y is also defined in 
appendix A. The coefficients frim are given by 



fllm^h['\kn) 



.f2lm — 



{kri]i\^\kri)y 
kri 



(71) 



where hj (a;) are the spherical Hankel functions of the first 
kind. It is readily seen that the operator J' has a non-empty 
nullspace (non-radiating sources). In particular, by writing 

V2mi{kr) = a{r)A2mi{r) + 5(r)A3„j(r), it follows that 

J {-b{r)A2rni{r) + a{r)A3mi{r)) = 0, where the orthonor- 

mality of the vector spherical harmonics (98) has been used. 

The adjoint operator J"* : H ^ O is readily obtained as 

2 oo I 



J*F{r)^-k^rjoJ2J2 E ^rim{kr)f: 



T—1 l — l m— — l 



f AU,^ir')-Fir')dS' 



(72) 



where r < tq, so that {JJ{r),F{r)) = {J{r),J*F{r)). 
The combined, self-adjoint operator J^J^* : "H ^- "H is then 
given by 

2 oo / „ 

J J* Fir) = kS'oJ2J2 E l/-""l' / \^ri,n{kr)\'dv 

T=l 1 = 1 m=-l •'^^a 

A^Ur) I Ali^ir') ■ F{r')AS\ 

(73) 
where the orthogonality of the regular vector spherical waves 
(99) has been used. It follows immediately from (73) that the 
vector spherical harmonics Arimi'i') are eigenvectors, and that 
the eigenvalues (the squared singular values) are given by 



where 



2 7 4 2 21 /• 12-2 



\Vrl7yi{kr)\'^dv. 



(74) 



(75) 



The eigenvectors of the operator J* J are similarly given by 
the regular vector spherical waves Vrim{kr) for r < rg, and 
the corresponding eigenvalues are again given by (74). 
The factor (75) can be evaluated explicitly as 



/•2 



^2ln, - 21 



ihikro) - ]i^i{kro)]i+i{kro)), 

— {{I + l)dl^i_^)rn + ^*l(; + l)m) 



(76) 



where the relation ]i{kr) = y^n / {2kr)Ji^i/2{kr) (where 
Jiy(-) is the Bessel function of order u) has been used together 
with the second Lommel integral [1, 17], as well as the 
recurrence relations for the spherical Bessel functions [28]. 
Note that the eigenvalues in (74) are independent of the m- 
index. 

The asymptotic behavior of the eigenvalues <7'^i„i in (74) 
for large values of I, can be analyzed by using 



11 fekr/2V 



1 1 fckr/2 



(77) 



see [28]. After some algebra, it is concluded that 

21 



rim 



/efcro/Sy 



^(7)' 



(78) 



where o(-) denotes the little ordo [27], and hence 



'^Tlm == k'^vlrjlfrlml^cr^l^ 



1 fekri/2 



I 



I 



-'' 'ekro/2Y A, 

-^ "(y) (79) 



ro 



n, 4^- 



It is concluded that cr^;^ 



as ? ^ 00, and the convergence 
is exponential. Hence, the operator J' is Hilbert-Schmidt. 

B. Spherically isotropic noise 

Spherically isotropic noise [4, 10] models a situation where 
the receiving sensors (antennas, microphones, etc.) are sub- 
jected to external noise consisting of plane waves impinging 
from arbitrary directions and with uncorrected amplitudes. 
The following electromagnetic model will be used here to 
model the spherically isotropic noise 

47r 



Es(r) 



Eo{k)c""'-''dn{k) 



(80) 



where il is the unit sphere, k the unit wave vector of the 
plane partial waves Es{r,k) = Eo(k)&''''"^ , and dQ{k) the 
differential solid angle. Here, Ef){k) = i?o(ai(fc)ei(fe) + 
Q:2(fc)e2(fc)) is modeled as a white (uncorrected) zero mean 
complex Gaussian stochastic process in the variable k, where 
Eq is a constant and k-Ef){k) ~ 0. Here, k, ei{k) and 62 (fc) 
are the unit vectors in the spherical coordinate system and 
ai{k) and a2(fc) the corresponding components of Eo{k). 
The covariance dyadic of Eo{k) is given by 

S{Eoik)E;{k')} = Ell2^2{k)5{k - k'), (81) 

where I2x2{k) = k x fcx is the projection dyadic perpen- 
dicular to k. 

The plane partial waves Es{r, k) can be expanded in regular 
vector spherical waves as 



Eo{k)e"'>''- ^J2J2 E <^rirnik>rUkr), 



(82) 



T—1 l—l m——l 



where the stochastic expansion coefficients are given by 

arimik) = 47ri'-"-iA;;„(fc) • Eoik), (83) 

see e.g., [3]. By exploiting the plane wave expansion (82), 
the covariance dyadic (81) as well as the orthonormality of 
the vector spherical harmonics (98), it is readily seen that the 
covariance dyadic of the spherically isotropic noise Es{r) is 
given by 

2 oo I 

T=l 1 = 1 m= — l 

(84) 

By using Vrim = {urim + Wri„i) /2, the dagger relations 

(100) as well as the expressions for the free-space Green's 

dyadic (101) and (102), it can be shown that the covariance 

dyadic (84) of the spherically isotropic noise is also given by 



1 



£{E,{r)Et{r')) ^ El-lui{G,{k,ry)) 



E^ 



1 



k 
1 \ sinfclr — r' 



I+^VV 1 



(85) 



^°47r V" ' fc2 ' V k\r~r'\ 
The observed noise field is defined here by 
N{r) ^-rxrx Es{nr) = 72x2(r) • E,{nr), (86) 
and the corresponding covariance dyadic is hence given by 

£{N{r)N*{r')} 

= l2^2{r)£{E,{nf)E:{nr')}l2^2{f') 



2 oo I 



(87) 



= ^oEE E 9limArUr)AU.^ir') 



r— 1 /—I ?n— - 



where 



giim =ji{kri), 

{krMkri))' 



(88) 



32/r 



kri 



where }i{x) are the spherical Bessel functions, and where (84) 
and (95) have been used. The covariance operator B : H ^ 
H for the Gaussian vector N{r) is defined by the property 

£-{(Fi(f),Ar(f))(F2(f),Ar(r))*} = {F,{f),BF2{f)), and 
is obtained as 



2 oo I 



BF{f) = ElY.Y. E 9li„^A,Ur) 



T=l 1 = 1 m=-l 



f AU^if')-F{f')dS'. 



(89) 



It follows immediately from (89) that the vector spherical 
harmonics A^(,„(r) are eigenvectors, and that the eigenvalues 
are given by 



Kim — -E'o'"l5'r/; 



(90) 



Note that the eigenvalues in (90) are idependent of the m- 
index. The asymptotic behavior of the eigenvalues Xrim for 
large values of / can be analyzed by using (77), which yields 

, 1 fekri/2V' 

Awm-feryf— ^j , (91) 



where br is a constant. It is concluded that Xrim — > as 
I — ?> oo, and the convergence is faster than exponential. Hence, 
the covariance operator B is trace class. 

C. Fisher information and the Cramer-Rao lower bound 

It is observed that the singular vectors Ui of the forward 
operator J , and the eigenvectors (pj of the covariance operator 
B coincide here with the vector spherical harmonics Arimi'f'). 
Hence, the situation is as described in section II-E above. The 
Cramer-Rao lower bound (64) is given by 

2 L 



cmL)^Y.Y.Y. 



XtI' 



T=l 1=1 



9 ' 
_l rim 



(92) 



where the eigenvalues are organized according to increasing 
/-index (multipole order). 

The asymptotics of the singular values cr^;„j of the forward 
operator J', and the eigenvalues Xrim of the covariance oper- 
ator B have been given in (79) and (91) above, respectively. 
The asymptotics of the eigenvalues of the Fisher information 
(63) is hence given by 

2/ / , \ 21 



rim 



Xrl 



I 

efcri/2 



»(i., 



(93) 



which implies that a'^i^^/Xrim -^ oo, and the Fisher infor- 
mation does not converge. However, the Cramer-Rao lower 
bound (92) converges to a finite value as L —>■ oo. 

In Fig. 2 a) is illustrated the convergence of the eigenvalues 
'^rim ^"'^ ^rim givcn by (74) and (90), respectively, and in 
Fig. 2 b) (the solid line) the convergence of the Cramer-Rao 
lower bound (92). Here, kro = 10, tq — 1, ri = 1.5 and 
^0 = 1. 



a) 10log{a^,i^,Xri„,} [dB] 



b) lOlogCRB(L) [dB] 
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Fig. 2. a) The singular values ct^;^ of the forward operator (upper blue 
lines) and the eigenvalues Xrim of the noise covariance (lower red lines). 
The solid and dashed lines correspond to t = 1 and t = 2, respectively, b) 
The Cramer-Rao lower bound CRB(L) for the electromagnetic inverse source 
problem with spherically isotropic noise. The solid (blue) line con'esponds 
to spherically isotropic noise only, and the dashed (red) lines correspond to 
white noise added with (from lower to upper dashed line) lOlogWNR = 
-60,-20,20 dB. 



It is emphasized that the spherically isotropic noise cor- 
responds to an "external" noise source. Suppose now that 
"internal" measurement errors are also present in the form 
of uncorrected measurement noise with variance a^,, which 



10 



is added to the eigenvalues Xrim, see also the noise model 
described in (55). The white-noise-ratio (WNR) is defined here 
by 

2 

WNR= '^ , (94) 

max Xrlm 
rim 

and the noise eigenvalues in (92) are then replaced as Xrim ~^ 
Xrim+'^w- In this case, the Fisher information (63) converges, 
and the Cramer-Rao lower bound (92) becomes infinite as 
L ^ oo. The situation is illustrated in Fig. 2 b) (the dashed 
lines) where 10 log WNR = -60,-20,20 dB. It is clear 
that if the white noise level is significantly lower than the 
maximum eigenvalue of the spherically isotropic noise, i.e., if 
cr^ << Tuaxrim Xrlm, then it is essential to incorporate the 
eigenvalues related to the spherically isotropic noise into the 
Fisher information analysis, and not only the singular values 
(Trim of the Jacobian J . 



IV. Summary and conclusions 

It is natural to consider an inverse imaging problem as 
an infinite-dimensional estimation problem based on a sta- 
tistical observation model. With Gaussian noise on infinite- 
dimensional Hilbert space, a trace class covariance operator 
and a Hilbert-Schmidt Jacobian, the appropriate space for 
defining the Fisher information operator is given by the 
Cameron-Martin space. A sufficient condition is given for the 
existence of a trace class Fisher information, which is based 
solely on the spectral properties of the covariance operator 
and of the Jacobian, respectively. Two important special cases 
arises: 1) The infinite-dimensional Fisher information operator 
exists and is trace class, and the corresponding pseudo-inverse 
(and the Cramer-Rao lower bound) exists only for finite- 
dimensional subspaces. 2) The infinite-dimensional pseudo- 
inverse (and the Cramer-Rao lower bound) exists, and the 
corresponding Fisher information operator exists only for 
finite-dimensional subspaces. An explicit example is given 
regarding an electromagnetic inverse source problem with 
"external" spherically isotropic noise, as well as "internal" 
additive uncorrected noise. 



Appendix 
The regular vector spherical waves are defined here by 

viimikr) = , V X {r}i{kr)Yi^{r)) 

V '(' + 1) 



= ii{kr)Aiim{f), 

V2im{kr) = -V X viim{kr) 
k 



(95) 



= 7 -A.2lm{r) + ylyl + 1)— 7 A3im(T-), 

where Arimi'i') are the vector spherical harmonics and j;(a;) 
the spherical Bessel functions, cf., [1,3,11,21,28]. The in- 
dices are given by / ~ 1, . . . , oo and m = —I, . . . ,1. The 



vector spherical harmonics Arim{r) are given by 

1 



Ailm{f) 



-.V X (rYimir)) , 



A2im{r) =r X Aiimir) 
A3im{r) ==rY;,„(r), 
where Yim{r) are the scalar spherical harmonics given by 



(96) 



Y 



Im \ 



'-""■>/^\/|^-'<»-)e-. 



(97) 

and where P™{x) are the associated Legendre functions [1]. 
The vector spherical harmonics are orthonormal on the unit 
sphere, and hence 



/ Ali^{f) ■ Ar'l'm'{f)dQ = Srr'Sll'S„ 
JSi 



(98) 



where 5*1 denotes the unit sphere, dfl — smdd6d(p and r — 
1,2,3. As a consequence, the regular vector spherical waves 
are orthogonal over a spherical volume Vr„ with 



v:;.i„,{kr)-Vr'i'm'ikr)dv 



(99) 



Srr'Sll'Smm' / \Vrlmikr)\ dv , 



where t — 1, 2. 

The out-going (radiating) and in-going vector spherical 
waves Urimikr) and Wrim{kr) are obtained by replacing the 
spherical Bessel functions ii{x) above for the spherical Hankel 
functions of the first and second kind, h\ '(x) and h\ (x), 
respectively, see [3,28]. The dagger notation {-j^ is used here 
to denote a sign-shift in the exponent of the factor c'™'*. Hence, 
for real arguments kr, it is observed that 



Kirnikr)=vli,Jkr), 

Kimikr)=wli^{kr), 

<imikr)^uli,^{kr), 



(100) 



where {•}* denotes the complex conjugate. 

The free-space Green's dyadic for the electric field satisfies 
V X V X Go(fc, r, r') - PG^ik, r, r') = IS{r - r'), where 
/ is the identity dyadic and 5{-) the Dirac delta function, and 
is given by 



Ge(fc,r,r') = (/+T^VV) 



I Qife|i — r'\ 



k^ Anlr — r' 



(101) 



see e.g., [3,11]. The free-space Green's dyadic can also be 
expanded in vector spherical waves as e.g.. 



2 CO ; 



Gc{k,r,r')^ikJ2Yl Yl '^rim{kr>)vli„Xkr<), 



T=l i=l m= — l 



(102) 

where r> (r<) denotes the vector in {r, r'} having the largest 
(smallest) length, cf., [3]. 
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